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ARITHMETIC CORRELATIONS OVER LARGE FINITE 

FIELDS 


J.P. KEATING AND E. RODITTY-GERSHON 


Abstract. The autocorrelations of arithmetic functions, such as the 
von Mangoldt function, the Mobius function and the divisor function, 
are the subject of classical problems in analytic number theory. The 
function field analogues of these problems have recently been resolved in 
the limit of large finite field size q. However, in this limit the correlations 
disappear: the arithmetic functions become uncorrelated. We compute 
averages of terms of lower order in q which detect correlations. Our 
results show that there is considerable cancellation in the averaging and 
have implications for the rate at which correlations disappear when q —> 
oo; in particular one cannot expect remainder terms that are of the order 
of the square-root of the main term in this context. 


1. Introduction 


The generalized twin-prime conjecture |9], predicts that for an r-tuple 
of distinct integers ai,... ,a r , which do not cover all residues modulo some 
prime p, there are infinitely many positive integers n such that n + a, are 
primes, for all 1 < i < r. In other words, for a = (ai,..., a r ) let 


(1) 


7t(.t; a) = #{1 < n < x\n + ai, ..., n + a r primes} 


then the conjecture says that 7r(x;a) -A oo as x —» oo, unless the local 
obstruction noted above holds. If ai, ... ,a r cover all residues modulo some 
prime p, then for any n there is an i such that p\n+a,i and 7r(x; a) is bounded 
ass A oo. Note that the number of primes up to x, i r(x), is equal to 7 r(x; 0), 
and the number of twin primes is 7r(x; 0, 2). 

The Hardy-Littlewood conjecture [9], gives the rate in which ir(x; a) tends 
to infinity: let 

(2) Y(p;a) = #{ai mod p,...,a r modp} 


and 


( 3 ) 



I r 1 - T( p;a)/p 

II (i-i / P y 
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then, unless a\,... ,a r cover all residues modulo some prime p, the product 
converges, i.e. C r (a ) > 0, and the conjecture predicts that 

x 


( 4 ) 


7r(x; a) ~ C r (a ) 


oo 


'log r x’ 

The Hardy-Littlewood conjecture can be rephrased using the von Mangoldt 
function 


(5) A (to) = 
as follows: 


logp if to = p k for some prime p and integer k > 1, 
0 otherwise 


( 6 ) 


E A(n + ai)--- A(n + o r ) ~ C r (a)x 

n<x 


There has recently been interest in the analogue of the Hardy-Littlewood 
conjecture over large finite fields. Let F 9 be a finite field of q elements, with 
q odd, and let F ? [T] be the ring of polynomials with coefficients in ¥ q . Let 
P n := {/ G F q [T] : deg / = n} be the set of polynomials of degree to, and 
M n := {/ G Fq[T] : deg / = n, /rnonic} be the set of monic polynomials of 
degree to. 

It follows from the work of Bary-Soroker [2] and Bender and Pollack [3], 
that for fixed n and in the limit of large (odd) q, the analogue of the Hardy- 
Littlewood conjecture holds in the form 

( 7 ) 4 E A (/)- A (/ + *0 = 1 + m, n, q ) 

Q f&M n 

with 

( 8 ) E(K,n,q)=0(q- 1 2) 


and where, by analogy with ([5]) 

(9) 

degp if / = p k for some prime polynomial p and integer k > 1, 
0 otherwise 


A (/) = 


and 0 / K e F ? [T], n > deg K. The bound (JHJ) results from using an 
algebraic irreducibility criterion and then invoking the Lang-Weil bound for 
the number of points on varieties in finite fields. It is significant that at 
leading order as q —>• oo, the von Mangoldt functions are uncorrelated in 
that the autocorrelation function is independent of I\. Information about 
any correlations is therefore contained in the error term E(K , to, q)\ however 
the bound (JHJ) is not sensitive enough to detect this. 

It is natural to speculate that the true order of E corresponds to square- 
root cancellation in the sum in dTJ), so that E(K,n,q) = 0(q ~ n / 2 ). Our 
results imply that this cannot, in fact, be the case. 

In this note, we study the sum of the error term E(K, to, q) over all monic 
polynomials K of a given degree. The method we use here is based on a 
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calculation of Keating and Rudnick [15] , who computed the variance of the 
sum of the von Mangoldt function over short intervals and in arithmetic 
progressions. Our approach applies as well to other arithmetic functions 
such as the Mobius function g(f) and the divisor function d(f). We begin 
by reviewing the result of m , which we shall need here. We then state our 
results for A(/). The extensions to g{f) and d(f) are outlined in section 4. 


1.1. Arithmetic progressions. For a polynomial Q £ F q [T] of a positive 
degree, and A £ F g [T] coprime to Q and any n > 0, set 

(10) tf(n;Q,A):= Y A (N) 

NeM n 
N=A mod Q 


The prime polynomial theorem in arithmetic progressions states that as 
n —» oo 


( 11 ) 


V(n;Q,A) ~ 


*(Q) 


where 4>(Q) is the Euler totient function for this context, namely the number 
of reduced residue classes modulo Q. Now set 

n n 

(12) G(n-Q):= Y | 2 

A mod Q ^ 

gcd(A,Q)=l 


It was shown in m that the following holds 


Theorem 1.1. In the limit q — >• oo, 

(13) = (degQ-l) + 0(T) 

where deg Q < n. 


The result is based on an equidistribution statement for the unitarized 
Frobenii of primitive odd characters [12] . 


1.2. Short intervals. For A £ P n of degree n, and h < n, a short interval 
of size h around A is defined by 

(14) I (A] h) := {f : \ \f - A\\ < q h } = A + P< h 

Where 11/|| := g de s/ and P<h = {0}uUo< m <fc -Fm is the space of polynomials 
of degree at most h. We have 

(15) #I(A-h)=q h+1 

Note that if /, g £ /(A; h), then there exists a polynomial of degree less than 
or equal to h such that / and g are congruent modulo this polynomial. For 
1 < h < n and A £ P n , set 

(16) v(A-h)= Y A (/) 

feI(A;h) 
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The mean value of v(A; h ), when we average over M n , is 

h)) := Y v ( A 'i h ) 

^ AeM n 

E a </) 

(17) 1 AeM n fei(A-,h) 

= 4'“ +1 e A </) 

y /eM„ 

= 

The last equality is due to the Prime Polynomial Theorem, which in this 
context is the identity 

(18) E A(/) = 9” 

/ EM-n 

Now, consider the limit as q -A oo of the variance of v(A\ h) 

(19) V{y(*-h)) = ^~ Y \v(A-,h) - (v(»]h)) | 2 

The following theorem was established in |l5j 
Theorem 1.2. In the limit q -A oo, and for h < n — 3, 

(20) -}—V(v(*\h)) =n-h -2 + 0(-^) 

The proof is based on an equidistribution statement for the unitarized 
Frobenii of primitive even characters P3I- 

1.3. Statement of results. The first quantity we study in this note is the 
sum of the error term E(K , n, q ), defined in (f8|). over all monic polynomials 
of a given degree: 

( 21 ) S E (k,n,q):= Y E ( K ,n,q) 

K0M k 

The second quantity we study is a ’’twisted” sum of E(K,n,q ): 

n —deg Q —1 

(22) S E (n, q; Q) = E E E{KQ,n,q) 

j =0 KeMj 

We have the following theorems for the above sums: 

Theorem 1.3. For k < n — 3 and in the limit q -A oo, 

(23) S E (k,n,q) = —^-b 0(-^) 

1 — q q A l A 
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Theorem 1.4. Let Q be a polynomial such that degQ < n - Then in the 
limit q —>• oo, 

(24) S E {n, q; Q ) = (n ~ deg ^ } + 0 ( J-) 

1 — q q A / z 

Corollary 1.5. For a polynomial Q of degree n — 1 the error term is 

(25) E (Q,n,q) = -^-j + 0( -^) 

We note that the number of terms in the sum in (|21l) is q k , so the average 
of E(K, n, q) over K E M*, is approximately — q^Tq-Tj' Similarly, the number 

of terms in the sum in ( 1221 ) is (/ _ 1 — where k = degQ, and so the average 
of E(KQ,n,q ) over K E M <n _^ is approximately - . 

Theorems o and o have implications for E(K,n,q ) that are worth 
noting here. First, they both involve sums of a large number of terms ( q k 
and 0(q n ~ k ~ 1 ) respectively). Therefore, the fact that both vanish as q —>• oo 
necessitates a considerable degree of cancellation. Second, setting k < n/2 — 
1, one sees that P51) is not consistent with E(K,n,q) being 0(q ~ n / 2 ), as 
might have been anticipated based on the (optimistic) assumption of square- 
root cancellation in the fluctuations in the sum in (|7|) . Indeed, setting k = 0, 
one sees that E( 1, n, q) = + 0(- 572 ) for all n > 4. The same implication 
follows from corollary 1 1.5 1 More generally, the maximum of \E(K, n, q)| with 
respect to K E M/ c is bounded from below by the modulus of the average of 
E, which is q k^_^ , and this is larger than q~ n / 2 for k < n/2 — 1. 


2. Sum of error terms 

In this section we use the result obtained in Theorem ll.2l to evaluate the 
sum of the error terms E(K , n, q). 

First, we use the definition of the variance of v(A; h ) to write 

(26) V(v(.; h)) = 4[ E ~ 2< l h+1 E vh ) + Q n Q 2(h+1) ] 

q AeM n AeM n 


Note that 
(27) 

AeM n 

For the sum involving v(A ; h) 2 we have 


E v(A;h)=q n (v(.-h))=q h+1+n 

AGMn 


(28) 


Y, v(A-,hf = y. E miws) 

A&M n A&M n f,g£l(A;h) 


Y Y A (/> 2 + E E A (/) A (») 


AeM n fei(A,h) 


A£M n f,g£l(A-,h) 
f^9 
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For the first term, we have (see Lemma 3.1 in m 
(29) 


E E A (/) 2 = qh+1 E A (/) 2 

A(E:Mn f G 

= q h+n+1 n + 0{n 2 q n / 2 ) 


For the second term, recall that if /, g £ I(A]h), then there exists a 
polynomial of degree smaller or equal to h such that / and g are congruent 
modulo this polynomial. The sum over f ^ g thus can be written as 
(30) 

E E E E a(/)a(/+j) = <,'“+■ EE E v/w+j) 


AeM n f£l(A;h) j= 0 deg J=j 
J^O 


/SM n j=0 deg J=j 
J^O 


We will restrict the J-sum to monies, multiplying it by g — 1, and so the 
right-hand side becomes 


(31) 


h +1 


(?-i)EEE A(/)A(/ + J) 


f€M n j= 0 JeMj 


Combining the above, we get 
(32) 


y«^ h )) h +1 
- n 9 


+ E E E A(/)A(/ + j) + 0( 1 

q feMn j =o jgMj 


qn/2-\-h-\-\' 


By the definition 0 of E{ J, n. g), we have 
(33) 

V(v(»]h)) 


q 


h +1 


= n — 


= n — 


q h+1 + (g - 1) E E( 1+ E ( J ’ n ’ q )) + °( 


j= 0 JgMj- 


gn/2+/i+l 


) 


1 + (g - i) E E n ’ q ) + 

j=o JeMj 


1 


(34) 


qn/2+h+l 

Now, we combine the expansion of V{v{»\ h)) with Theorem 11.21 to write 

h 

1 , 


(g-i)E E E(J,n,q)=-{h+l) + 0( 


j =0 J&Mj 


Vq 


By subtracting the sum up to h — 1 from the sum up to h, we get Theorem 

P 

(g-i) E = - 1 + o(E) 


(35) 
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Thus the average of the error terms is 

(36) \ y E(J,n,q) = — --r —r + 0{ . ) 

v ; q h tt„ (q-i)q h V +3/2 


3. Twisted sum 


In this section, we use Theorem 11.11 to evaluate the sum of E(KQ,n,q) 
for fixed Q with K € Af <n _ c ] eg Q. First, we use the definition of G(n; Q ) to 
write 


(37) 

G(n;Q) 


E 2 


gcd(A,Q)=l 


HQ) 


E '&(n;Q,A) + 


Jin 


gcd(A,Q)=l 


HQ) 


Note that 


(38) 


E E A (/) 

gcd(A,Q)=l deg f=n 

gcd(/,Q)=l 

= E A </> - E v/) 

deg f=n deg f=n 

deggcd(/,Q)>0 

= q n - E degP 

deg P|n 

P|Q 

prime 

= q n + 0(deg Q) 


For the sum over 'F(n; Q, A) 2 , we have 


(39) 


E HmQ,A ) 2 

gcd(A,Q)=l 


E Mf)Ms) 

f,g£M n 
f—g mod Q 
gcd(/,Q)=l 


E A </) 2 + E A (/) A <») 


f £.M n f\g£Mn 

gcd(/,Q)=l f=g mod Q 

gcd(/,Q)=l 

f^g 


For the first term, we have (by Lemma 3.1 in [15] ) 


(40) 


E a(/) 2 = e A (/) 2 - E A (/) 5 

/ GAf n 


/G 

gcd(/,Q)=l 


deg f=n 
deg gcd(/,Q)>0 


= nq n + 0{n 2 q n ! 2 ) + 0(deg Q 2 ) 
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The sum over f ^ g can be written as in the appendix of [15] (equation 
A. 14) 

n —deg Q—l 

(41) E A(/)A( 9 ) = 0j — 1) E E E MDMl + JQ) 

f,geM n j =0 JdMj /£M„ 

f=g modQ 
gcd(/,Q)=l 

S+g 

Combining the above, we obtain 

n( ra-degQ-1 

-^ = ^(9-1) E E E a(/)a(/ + j<?) 

(42) q q 3= 0 JtMj f&M n 


+ » + o ( »V^ + o(^)-^ + o(^) 

By using the function field version of the Hardy-Littlewood conjecture, i.e. 
and noting that as q —>• oo 

n degQ 

(43) -a 1 


we have 


G(n;Q) 


(44) 


HQ) 

n —deg Q—l 

(9-1) E E (1 + E(JQ,n,q )) 

j =0 J&Mj 

+ n + 0(n 2 q~ n / 2 ) + 0(- j^) - q n ~ A ^ Q + O(^) 


n— deg Q—l 

= ( 9 - 1 ) ^ E(JQ,n,q) 

j =o JeMj 

+ „-i + o(„V"/ 2 ) + o(^) + 0( ^) 

Now, we combine the expansion of G(n',Q) with ( 1131 ) . obtaining 

n—deg Q—l 

(45) (q - 1 ) ^2 ^2 E (JQi n ,q) + n- 1 = (degQ - 1 ) + 0 ( —) 

i=o j^Mj vQ 

Therefore we have the following expression for the error term E(JQ,n, q): 

n —deg Q—l 

(46) (9-1) ^2 E ( J Qi n ,<l) =-(n-degQ)+ 0(—) 

j =0 J&Mj V 5 

which proves Theorem (jl.4jl . From this we can deduce that for a polynomial 
Q of degree n — 1 the error term is 

(q - 1 )E(Q,n,q) = -1 + 0 (^=) 

y/q 


( 47 ) 













ARITHMETIC CORRELATIONS OVER LARGE FINITE FIELDS 


9 


and that in general the average is ^-degO-i + Of ^-deg 0+1/2 )> because the 

_____ < n — deg Q 1 

number of terms in the sum in (1461) is -——^-. 

4. Further examples - the Mobius function and the divisor 

FUNCTION 

In the following section we use the method demonstrated above to eval¬ 
uate sums of the error terms in two other important problems. The first is 
the additive divisor problem over ¥ q [T] (see PPj), and the second is Chowla’s 
conjecture over F 9 [T] (see [5]). 

4.1. Definitions. For an arithmetic function a we define 
(48) v a (A;h)= Y a(f) 

f£l(A;h) 

The mean value of v a (A\ h ) is therefore 

(vafah)) = — Y v <*(A;h) 


(49) 


AeM n 
= -q h+1 


Y 


f(zM n 


= q h+1 ( a) n 

The variance of v a (A; h ) is given by 

(50) V(v a (»\h)) := Y \ v <*{A\h) - {v a (»;h))\ 2 


q“ 


AeM n 


In the following we use the function field zeta function 

(«) OM = 737^ 

which we also write as Z(u) = (1 — qu )~ x by setting u = q~ s . 

4.2. The divisor function. The additive divisor problem over Z (some¬ 
times called ’’shifted divisor” or ’’shifted convolution” problem), concerns 
the asymptotics of the sum 


(52) 


D(x; h ) := Y^ d(n)d(n + h) 


n<x 


where d is the divisor function. Ingham HH computed the leading term, 
and Estermann | 8 ] gave an asymptotic expansion 


(53) 

where 

(54) 


Y2 d(n)d(n + h) = xP 2 i}ogx\ h ) + 0(x 12 (logx) 3 ) 


n<x 


P 2 (u;h) = -^yo-_i(/i)u 2 + ai(h)u + a 2 {h) 
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with 

(55) cr w (h) = ^2k w 

k\h 

and a\(h), 02 (h) are complicated coefficients. 

The size of the reminder term plays an important role in various problems 
in analytic number theory; see, for example, 0 , m- 

Andrade, Bary-Soroker and Rudnick [1] studied the additive divisor prob¬ 
lem over ¥ q [T ], showing that in the limit q —>• oo 

(56) 4 E d U) d U + J ) = (n + 1) 2 + E d (J,n,q) 

q f&M n 

when 0 ^ Js F g [T], and deg(J) < n, with 

(57) E d {J,n,q) = 0(q-*) 

This corresponds to a(f) = d(f ) in the definition above. As before, we will 
use a result that is based on an equidistribution statement for the unitarized 
Frobenii of primitive even characters in order to study the sum of E(J,n, q) 
over monic polynomials of a given degree. To this end, we quote the following 
theorem from [14] : 


Theorem 4.1. In the 

If0<h<^—2 then 

(58) V(v d (»\ 
If h = | — 1 then 

(59) 

If § < h < n then 

(60) 


limit of large field size, q —>• oo, the following holds: 


— (n — 2h + l 


h)) = q h 


Di+1 


+ o(V) 


V(v d (.-h)) = 0(- 


Ji+i 


Vi 

V(v d (*\h)) = 0 


We use the definition of the variance of V(v d (»; h)) to obtain 
(61) 

1 h 

V{v d {*;h)) = —[q h+1 E d(f) 2 + q h+1 E E E d VW + J ) 

^ fknU *£j=i 

J +0 

_ 2q 2(h + 1 ) q n {{d)n) 2 + q 2 {h+ l) {{ d)n) 2 ] 

By considering the generating functions of YlfeM n ^(/) aR d °f d(/) 2 , 

which are Z(u) 2 and ^(S) res P ec bi v ely, we have that 

£ nn = «>+i) 

feM n 


( 62 ) 
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and 

(63) X d (-f) 2 = q 

feM n 

Thus the variance of V(vd(*‘, h)) is 

(64) 


n + 3 
3 


_i(n + 1 


l f 


n + 3\ fn + 1 
3 )~ q 


-q h+1 [n 


+1 ) 2 +:77 d (/M/+ J )l 


f&Mn j =0 deg J=j 
J +0 


which by ([56ft is 
(65) q h+1 


n + 3 
3 


_ 1 (n+ 1 


9 ‘I o ) -(n+ l) 2 + (g-l) X X E d(J,n,q)} 

j =o JeMj 

By combining the above with Theorem (14.11) . and subtracting the sum up 
to h — 1 from the sum up to h, we have 

Theorem 4.2. In the limit q — >• oo : 

IfO<h<% — 2 then 

(66) X E d( J - n ’ ( l) : 

J&M h 

If h = Tj — 1 f/ien 

1 


n — 2h — 1 
3 


n — 2/i + 1 

3 


1 


+ o(— 


(67) 

// ^ < h < n then 

( 68 ) 


2 W,n,g) = -l + 0( 






X] E d(J,n,q) = 0 

J&M h 


We note that the number of terms in the sum over J £ is q h , and so 
Theorem 14.21 determines the average of Ed when both sides of the equation 
are divided by q h . As for the von Mangoldt function, this theorem demon¬ 
strates considerable cancellation when Ed(J,n,q ) is summed over J, and 
establishes a lower bound on its size which rules out the optimistic guess 
that Ed(J,n,q) = 0{q ~ n / 2 ). 

4.3. The Mobius function. Chowla’s conjecture j6] asserts that given an 
r-tuple of distinct integers or,..., a n , and e* £ {1,2}, not all even, then 


(69) 


1 


lim — V /j,(n + aiY 1 ■ ■ ■ /x(n + a r Y r = 0 
IV—> 0 o N 


n<N 


This conjecture has recently been shown by Sarnak to imply that /x(n) does 
not correlate with any zero entropy sequence m- 
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Carmon and Rudnick [5], proved the function field version of the Chowla 
conjecture, in the limit of large (odd) field size. The extension to even 
characteristics has been established by Carmon in [4]. Here the Chowla 
conjecture, is shown to hold in the form 

(70) | Y, fi(f + «i) €l • • • n(f + o-rY r \< 2mq n ~ 1 ^ 2 + 2>m 2 q n ~ 1 

f GAf n 

where r > 1, n > 1 (in the case of even characteristics n > 2) and a±,... ,a n € 
F 9 [T] are distinct polynomials with deg ay < n. As before, e* € {1,2}, not 
all even. 

We will focus on the case of r = 2, a± = 0, and ei, €2 = 1. Denote by 

(71) Ep(J, n, q) := \ Y M/M/ + J) 

Q f&M n 

As before, we will use a result that is based on an equidistribution statement 
for the unitarized Frobenii of primitive even characters, in order to study 
the sum of E^(J,n,q) over monic polynomials of a given degree. To this 
end, we quote the following theorem from m 

Theorem 4.3. In the limit of large field size, q —>■ oo, and for h < n — 4, 

(72) V{v,{.-h))=q h+1 + 0(q h+1 / 2 ) 

Next, we use the definition of the variance of V{v^{»] h )) to obtain 

(73) 

V(v„(.-,h ,)) = 4b' ,+1 E + EEE M/M/ + J) 

q feM n feM n j=0 deg J=j 

-2<M + V(( fJt)n) 2 +q 2(h+l)^ fJl)n) 2 ] 

The analysis in this case follows exactly the same lines as in the previous cal¬ 
culations and so we omit the details. By considering the generating functions 
of E/e«„ M/) and of E/ G M n M/) 2 > which are and respectively, 
we conclude that for n > 1 

(74) Y M/) = 0 

feMn 


(75) Y M/) 2 

f&M n 

Thus the variance of R(ud(«; h)) is 


Cj(2) 


(76) 


h+1 \ 


Cg(2) 


+ (q-i)Y 


j =0 deg J=j 


By combining the above with Theorem (14.31) . and subtracting the sum up 
to h — 1 from the sum up to h, we have 
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Theorem 4.4. In the limit q —>• oo, and for h < n — 4, 

(77) | ^ ^(J,n,g)| = O(^) 

deg J=h ^ 
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